Imaging system for vehicle

ABSTRACT

A vehicular imaging system includes a camera with an imaging sensor. A first spectral filter is disposed at each pixel of a first set of photosensing pixels, a second spectral filter is disposed at each pixel of a second set of photosensing pixels, and each pixel of a third set of photosensing pixels senses at least near infrared radiation. Image data provided to the image processor includes output of the first set of photosensing pixels, output of the second set of photosensing pixels and output of the third set of photosensing pixels. The outputs of the photosensing pixels of the first and second sets of photosensing pixels are processed for a color camera function of the vehicle. The output of the photosensing pixels of the third set of photosensing pixels is processed for a night vision function of the vehicle.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 16/010,581, filed Jun. 18, 2018, now U.S. Pat. No. 10,616,507,which is a continuation of U.S. patent application Ser. No. 14/562,826,filed Dec. 8, 2014, now U.S. Pat. No. 10,003,755, which is acontinuation of U.S. patent application Ser. No. 13/896,845, filed May17, 2013, now U.S. Pat. No. 8,908,040, which is a continuation of U.S.patent application Ser. No. 12/681,204, filed May 28, 2010, now U.S.Pat. No. 8,446,470, which is a 371 national phase filing of PCTApplication No. PCT/US08/78700, filed Oct. 3, 2008, which claims thebenefit of U.S. provisional application Ser. No. 60/977,430, filed Oct.4, 2007, which is hereby incorporated herein by reference in itsentirety.

FIELD OF THE INVENTION

The present invention relates to vision systems for vehicles and, moreparticularly, to an imaging sensor or camera for a vision or imagingsystem of a vehicle.

BACKGROUND OF THE INVENTION

Traditional color imagers or image sensors or pixelated imaging arraysuse a Bayer pattern of pixels and pixel filters, which has ared-green-green-blue (R-G-G-B) pixel/filter configuration (such as shownin FIG. 1). In such a pixelated array, the sensor includes individualoptical filters that transmit red, green or blue colors and that aredisposed at or coated on the individual pixels. Thus, there is a “redpixel” 12 a, a “blue pixel” 12 b and two “green pixels” 12 c arranged toform a 2×2 sub-array 10 that is repeated over the pixelated array.

The three color filters (R, G, and B) not only pass ranges ofwavelengths or spectral bands that are corresponding to red, green andblue colors, they also pass through a significant amount of wavelengthsin the infrared (IR) or near infrared (NIR) region or band of thespectrum. Therefore, the color imager sensitivity or quantum efficiencyspectrum typically has a rich IR or NIR response even with the R, G, andB color pixels. For example, a typical silicon CMOS color sensor'sspectrum response is shown in FIG. 2. The IR response of the R, G and Bpixels is comparable or higher than the pixels' response of visiblespectrum. The IR light from the environment thus may wash-out the colorresponse in the visible spectrum and thus may distort the image colorreproduction. This is often referred to as IR contamination. In atraditional color camera, in order to reproduce a true color image, anIR cut-off filter is usually used to cut off or reduce light or energyat or in the IR band or region of the spectrum so as to allow only (orsubstantially only) the visible light to pass through the filter so asto be imaged by the RGGB pixels, in order to reduce or limit orsubstantially eliminate the IR contamination. Such an IR cut-off filteris typically made of multilayer coatings on a glass or plastic element,such as a flat glass plate, that is added on to a lens assembly of theimager or onto a surface of a lens element of the lens assembly of theimager. The coating process and added material increase the cost of thelens, sometimes significantly.

On the other hand, for IR imaging applications that have the mainspectrum of interest in the NIR region of the spectrum, where siliconCCD or CMOS can provide good quantum efficiency, one may need to cut thevisible spectrum off from the light reaching the imager. A long passfilter that is a multilayer coating on a glass or plastic flat plate istypically used to let through NIR light only. Security cameras and somespecial industrial machine vision systems are among the applications ofsuch a filter-imager configuration. For cameras that operate in both dayand night for such applications, mechanical switches of IR cutofffilters and IR pass filters are used on silicon CMOS or CCD cameras. Theadditional cost of two filters and mechanical switches, as well as thereliability of the moving mechanical switch, make such systemsundesirable, and such a design is not suitable for automotive orconsumer digital cameras or cell phone cameras.

SUMMARY OF THE INVENTION

The present invention provides an imaging sensor or camera having apixel configuration that combines red (R), green (G), and blue (B)sensing pixels with an infrared (IR) sensing pixel. The presentinvention provides de-mosaicing techniques for processing the R-G-B-IRimager pixels. The imaging sensor or camera of the present invention issuitable for use in automotive camera products, such as for vision-basedsystems, such as headlamp control systems, reverse aid systems, lanedeparture warning systems, object detection systems, rain sensorsystems, front collision warning, night vision and/or the like. Theimaging sensor or camera of the present invention may also be suitablefor use in security cameras, consumer digital cameras, cell phonecameras and industrial machine vision systems and/or the like.

Therefore, the imaging sensor of the present invention can provide bothvisible and IR imaging capability without external filters andshuttering/switching mechanism. Such a pixel configuration of R-G-B-IRpixels is applicable to various pixelated imaging array type sensors,such as CMOS, CCD and/or other image sensing devices. The imaging sensorof the present invention is suitable for applications in securitycameras, automotive cameras, consumer digital cameras, cell phonecameras, and industrial machine vision cameras, as well as other marketsand/or applications.

These and other objects, advantages, purposes and features of thepresent invention will become more apparent upon review of the followingspecification in conjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic of a typical R-G-G-B pattern or Bayer pattern;

FIG. 2 is a schematic of a silicon CMOS color sensor's spectrumresponse;

FIGS. 3A and 3B are schematics of R-G-B-IR pixel patterns in accordancewith the present invention;

FIG. 4 is a schematic of an array having a R-G-B-IR pixel pattern inaccordance with the present invention;

FIG. 5 is a schematic of a 2×2 window de-mosaicing method in accordancewith the present invention;

FIG. 6 is a schematic of a 2×2 rolling window de-mosaicing method inaccordance with the present invention;

FIG. 7 is a schematic of a 3×3 rolling window de-mosaicing method inaccordance with the present invention;

FIG. 8 is a schematic of a camera system combining forward viewingvision and NIR night vision display in accordance with the presentinvention;

FIG. 9 is a schematic of a rear viewing camera system with enhanced lowlight sensitivity, hardware switches normal color image and low lightNIR imaging in accordance with the present invention;

FIG. 10 is a schematic of a de-mosaicing technique for low light imagingin accordance with the present invention; and

FIG. 11 is a schematic of a rear viewing camera and display fordisplaying color images under normal lighting conditions and/ormonochrome images under low lighting conditions in accordance with thepresent invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Imaging Sensor Description

The imaging sensor of the present invention comprises a two-dimensionalpixelated imaging array having a plurality of photo-sensing pixelsarranged or disposed or established on a semiconductor substrate. Forexample, the imaging sensor may comprise acomplementary-metal-oxide-semiconductor (CMOS) or a CCD imaging sensoror device or the like, and may utilize aspects of the imaging sensorsdescribed in U.S. Pat. Nos. 5,796,094; 6,097,023; 6,320,176; 6,313,454;6,559,435; 6,831,261; 6,396,397; 5,877,897; 6,498,620; 5,670,935;6,806,452; 6,946,978; 7,123,168; 7,004,606; 7,005,974 and/or 5,550,677,and/or PCT Application No. PCT/US07/75702, filed Aug. 10, 2007 andpublished Feb. 28, 2008 as International Publication No. WO 2008/024639,and/or U.S. patent application Ser. No. 11/239,980, filed Sep. 30, 2005,now U.S. Pat. No. 7,881,496; Ser. No. 11/105,757, filed Apr. 14, 2005,now U.S. Pat. No. 7,526,103; and/or Ser. No. 10/534,632, filed May 11,2005, now U.S. Pat. No. 7,965,336; and/or PCT Application No.PCT/US03/36177, filed Nov. 14, 2003 and published Jun. 3, 2004 asInternational Publication No. WO 2004/047421; and/or PCT Application No.PCT/US06/41709, filed Oct. 27, 2006 and published May 10, 2007 asInternational Publication No. WO 07/053404, which are herebyincorporated herein by reference in their entireties.

The arrangement of pixels has a pixel pattern that is derived from theBayer pattern, but has one of the green filters or pixels replaced by aninfrared (IR) filter or pixel 12 d, such as shown in FIGS. 3A and 3B.The IR filter or pixel has an optical filter that limits or reduces orsubstantially precludes light or electromagnetic energy in the visibleband or region of the spectrum so as to pass or substantially pass IRenergy (for example, energy having a wavelength at or above about 700 nmor thereabouts) so that the pixel principally senses IR energy or NIRenergy. The red, green and blue pixels may have a similar filter asimplemented for typical Bayer pattern imaging sensors (and thusprincipally sense the respective visible color band of the spectrum),while the IR pixel has a filter that cuts off light in the visiblespectrum and passes only light in the NIR or IR band or region of thespectrum. Optionally, for example, the IR filter may comprise a redfilter and a blue filter stacked one on top of the other to form a NIRor IR pass filter. The system may function to process an output of theNIR or IR pixel to determine an infrared component of the imaged sceneand may subtract the infrared component from the outputs of the red,green and blue pixels to obtain a more accurate or true color responsefor the pixels and to limit or avoid infrared color wash-out, asdiscussed below.

Optionally, the R-G-B-IR filter arrangement may be any suitablearrangement of pixels/filters in a repeating pattern, such as the 2×2arrangements of pixels shown in FIGS. 3A and 3B. As shown in FIG. 3A, a2×2 pixel matrix or sub-array may have the IR pixel 12 d positioneddiagonal to the red pixel 12 a. Since the red pixel has a stronger NIRresponse than the green pixel 12 c and the blue pixel 12 b, such anarrangement will generate a more evenly distributed IR image by placingthe IR pixel and the red pixel diagonally. Such an arrangement will alsobenefit the IR image resolution in the de-mosaicing methods describedbelow. Alternately, and as shown in FIG. 3B, the pixel arrangement mayhave a repeating 2×2 matrix or sub-array with the green and blue pixels12 c, 12 b along one column and the red and IR pixels 12 a, 12 d alongthe other column, such that half of the columns (green and blue pixelsonly) has a weaker IR sensitivity, while the other half of the columnshas a greater IR sensitivity. The benefit of such an arrangement is thedirect replacement (as compared to a typically R-G-G-B pixelarrangement) of a green pixel by an IR pixel (because a typicallyR-G-G-B pixel arrangement would have the IR pixel replaced with a greenpixel), which results in smaller imager fabrication process change. Inthe de-mosaicing technique discussions below, the de-mosaicingtechniques are described in relation to the pixel arrangement of FIG.3A, but it is envisioned that similar de-mosaicing techniques may becarried out with the pixel arrangement of FIG. 3B, while remainingwithin the spirit and scope of the present invention.

Since the red, green and blue pixels typically may have a similar IRresponse as the IR pixel, the establishment of the IR pixel on eachsub-array of the pixelated array allows for a determination of the IRcomponent present in the light being imaged. Thus, the IR component (theIR or NIR energy or light sensed by the IR pixel) can be subtracted fromthe red, green and blue pixel response signal to obtain a more accurateor true color response for the pixels and to limit or avoid IR colorwash-out. Further, with the IR cut-off in the sensor pixel level, thereis no need to add IR cut-off filter in a lens. This simplifies the lensdesign and results in significant cost savings in manufacturing of theoptical lens for the imager. Because the sensitivities in the IR regionof the red, green and blue pixels may be different, factors or ratiosmay be applied to the IR signal when subtracting the IR value orcomponent from the color signals. For example, the color values of thepixel signals may be determined via the following equations:Red Color Value=Red Pixel Value−A _(R) *IRPixel Value;Green Color Value=Green Pixel Value−A _(G) *IRPixel Value;Blue Color Value=Blue Pixel Value−A _(B) *IRPixel Value; andIR Value=IR Pixel Value,where A_(R), A_(G), and A_(B) are the selected factors for the redpixels, green pixels and blue pixels, respectively.

Optionally, for example, these factors can be selected to be 1 tosimplify the de-mosaicing circuitry or software. However, the spectralresponses of the R, G and B pixels in the NIR region may not match wellwith that of the IR pixel. As a result, it may be difficult to totallycancel out the IR signal from the R, G and B pixels that sensing lightin the visible range of the spectrum. Optionally, the factors of each R,G and B color can be determined by the corresponding spectralsensitivities of the sensor. The spectrum contribution of the lightsource should also be considered. For example, for applications wherethe scene is illuminated by a light source or light sources emittingillumination within a known spectral band or region of the spectrum, acombination of the light source and the imager's spectral sensitivitymay determine the factors. For example, if an imaged object orillumination source contains narrow-band IR illumination around 750 nm,the imager with spectral characteristics such as shown in FIG. 2 shouldhave different green and blue factor values. As another example, when anincandescent light source or a Halogen light source is used, because oftheir light output being in a more wide spread spectrum in both visibleand IR ranges, there should be another set of factors for the red, greenand blue pixels. Because of the known source spectrum in such cases, thefactors can be pre-determined. Also, if an application involves multiplecontrolled illumination spectra sources (with at least one of theillumination sources generating a light output having different spectralcharacteristics than at least one other of the illumination sources),the factors corresponding to each illumination source can be stored inthe memory of the imager or camera or processor and loaded to the imageror the processor for calibration when the illumination source changes toa different illumination source generating a light output having adifferent spectral characteristic.

Optionally, for illumination sources that generate a light or energyoutput with changing or un-controlled spectrum or spectralcharacteristics, such as sun light at different times of the day, moresophistical image processing can be applied to determine the factors.For example, an iterative approach can be used in which multiple passeswith different factors determine the best factors. As another example, acolor conversion matrix can be used in a manner that is similar to anauto white balance technique that is commonly used in visible lightcameras, wherein a statistical analysis of the scene image is performedto determine the contribution of individual visible colors and the IRspectrum and to determine the values of three factors.

With the pixel arrangements discussed above and the de-mosaicingtechniques described below, an imager or imaging system of the presentinvention can produce a visible color image and a NIR image at the sametime. The following sections describe the different de-mosaicingtechniques of the present invention.

De-Mosaicing Techniques

A pixelated imaging array of the present invention may have an R-G-B-IRpixel pattern, such as described above and shown in FIG. 3A. Althoughshown in FIGS. 4-7 as a 4×4 array of pixels, clearly the imaging arraymay have and preferably has an array of at least about 40×40 pixels andmay have an array of, for example, about 512×512 pixels or 640×480pixels or thereabouts or any other suitable array for capturing imagesin the field of view of the imaging sensor. In the illustratedembodiments, the 4×4 pixel sub-arrays are described below with theindividual pixels at respective coordinates corresponding to the row andcolumn location of the pixels (for example, the R1 red pixel is atlocation r0,c0, while the G3 green pixel is at location r2,c1, and soon). The de-mosaicing techniques of the present invention are applicableto any sized pixelated array and repeat the process over the entirearray to de-mosaicize the pixelated array. The following de-mosaicingformulas describe different methods of extracting red, green, blue andIR values for each pixel from itself and the adjacent pixels. Thus, avisible color image and a NIR image can be produced at the same time viathe processing techniques of the present invention.

2×2 Window De-Mosaicing

For an R-G-B-IR pixel pattern, each pixel of the array has itsassociated primary color (or is sensitive to a particular band of thespectrum and is less sensitive to other bands of the spectrum), whichcomes from its filter type. For example, and with reference to FIG. 5,the R1 red pixel at r0c0 has a primary color of red, so that the redvalue for this pixel is readily available, while the G1 green pixel atr0,c1 has a primary color of green, so that the green value for thispixel is readily available, and the B1 blue pixel at r1,c0 has a primarycolor of blue so that the blue value for this pixel is readilyavailable. The green, blue and IR values for this pixel location arecalculated from the adjacent pixels in a 2×2 window. Each window coversfour pixels in two rows and two columns and in order of red, green, blueand IR, such as shown in FIG. 5. The 2×2 window does not overlap anyadjacent 2×2 windows. As an example in the first 2×2 window, theformulas for calculating the color or IR values for the pixel locationsare as follows:R(r0c0)=R1−A _(R) *IR1,G(r0c0)=G1−A _(G) *IR1,B(r0c0)=B1−A _(B) *IR1,and IR(r0c0)=IR1R(r0c1)=R1−A _(R) *IR1,G(r0c1)=G1−A _(G) *IR1,B(r0c1)=B1−A _(B) *IR1,and IR(r0c1)=IR1;R(r1c0)=R1−A _(R) *IR1,G(r1c0)=G1−A _(G) *IR1,B(r1c0)=B1−A _(B) *IR1,and IR(r1c0)=IR1; andR(r1c1)=R1−A _(R) *IR1,G(r1c1)=G1−A _(G) *IR1,B(r1c1)=B1−A _(B) *IR1,and IR(r1c1)=IR1,where A_(R), A_(G), and A_(B) are the selected factors for the redpixels, green pixels and blue pixels, respectively, and where R(rx,cx)is the red color value for the respective pixel location, G(rx,cx) isthe green color value for the respective pixel location, B(rx,cx) is theblue color value for the respective pixel location, and IR(rx,cx) is theinfrared value for the respective pixel location, and where R1 is anoutput value of the red pixel of the window or sub-array, G1 is anoutput value of the green pixel of the window or sub-array, B1 is anoutput value of the blue pixel of the window or sub-array, and IR1 is anoutput value of the IR pixel of the window or sub-array.

Thus, the value of the IR component for that pixel location (asdetermined by multiplying the value of the IR pixel by a factoring valuefor each color) is subtracted from the value of the respective colorpixel (and the color for the adjacent pixels also has the factored IRvalue subtracted therefrom) to determine the color value of each pixellocation. This is repeated for each 2×2 window of the array.

2×2 Rolling Window De-Mosaicing

Unlike the 2×2 Window De-mosaicing method above, the 2×2 Rolling WindowDe-mosaicing process shifts the 2×2 window by one pixel at a time inboth the row and column directions, such as can be seen with referenceto FIG. 6. The following formulas show an example of de-mosaicing fourpixels of each color band or IR band:R(r0c0)=R1−A _(R) *IR1,G(r0c0)=G1−A _(G) *IR1,B(r0c0)=B1−A _(B) *IR1,and IR(r0c0)=IR1R(r0c1)=R2−A _(R) *IR1,G(r0c1)=G1−A _(G) *IR1,B(r0c1)=B2−A _(B) *IR1,and IR(r0c1)=IR1R(r1c0)=R3−A _(R) *IR1,G(r1c0)=G3−A _(G) *IR1,B(r1c0)=B1−A _(B) *IR1,and IR(r1c0)=IR1; andR(r1c1)=R4−A _(R) *IR1,G(r1c1)=G3−A _(G) *IR1,B(r1c1)=B2−A _(B) *IR1,and IR(r1c1)=IR1,where A_(R), A_(G), and A_(B) are the selected factors for the redpixels, green pixels and blue pixels, respectively, and where R(rx,cx)is the red color value for the respective pixel location, G(rx,cx) isthe green color value for the respective pixel location, B(rx,cx) is theblue color value for the respective pixel location, and IR(rx,cx) is theinfrared value for the respective pixel location, and where Rx is anoutput value of the red pixel of a respective sub-array, Gx is an outputvalue of the green pixel of a respective sub-array, Bx is an outputvalue of the blue pixel of a respective sub-array and IRx is an outputvalue of the IR pixel of a respective sub-array.

Thus, the color values for the individual pixel locations are based onthe color values for the respective color pixel within the particular2×2 window, as reduced by the factored IR value of the IR pixel withinthe particular 2×2 window. The 2×2 window moves or shifts along the rowsand columns during the processing of the pixelated array.

3×3 Rolling Window De-Mosaicing

Optionally, the imaging array may be de-mosaiced via a 3×3 rollingwindow de-mosaicing technique. In this method, each pixel will have itsred, green, blue and IR information from averaging the adjacent pixelsexcept for the primary color itself, such as can be seen with referenceto FIG. 7. For example, the formulas for the pixel values at the pixellocations r1c1, r1c2, r2c1 and r2c2 are:R(r1c1)=(R1+R2+R3+R4)/4−A _(R) *IR1,G(r1c1)=(G1+G3)/2−A _(G)*IR1,B(r1c1)=(B1+B2)/2−A _(B) *IR1, and IR(r0c0)=IR1;R(r1c2)=(R2+R4)/2−A _(R)*(IR1+IR2)/2,G(r1c2)=(G1+G2+G3+G4)/4−A_(G)*(IR1+IR2)/2,B(r0c1)=B2−A _(B)*(IR1+IR2)/2, andIR(r0c1)=(IR1+IR2)/2;R(r2c1)=(R3+R4)/2−A _(R)*(IR1+IR3)/2,G(r2c1)=G3−A_(G)*(IR1+IR3)/2,B(r2c1)=(B1+B2+B3+B4)/4−A _(B)*(IR1+IR3)/2, andIR(r2c1)=(IR1+IR3)/2; andR(r2c2)=R4−A _(R)*(IR1+IR2+IR3+IR4)/4,G(r1c1)=(G3+G4)/2−A_(G)*(IR1+IR2+IR3+IR4)/4,B(r2c2)=(B2+B4)/2−A _(B)*(IR1+IR2+IR3+IR4)/4,and IR(r2c2)=(IR1+IR2+IR3+IR4)/4,where A_(R), A_(G), and A_(B) are the selected factors for the redpixels, green pixels and blue pixels, respectively, and where R(rx,cx)is the red color value for the respective pixel location, G(rx,cx) isthe green color value for the respective pixel location, B(rx,cx) is theblue color value for the respective pixel location, and IR(rx,cx) is theinfrared value for the respective pixel location, and where Rx is anoutput value of the red pixel of a respective sub-array, Gx is an outputvalue of the green pixel of a respective sub-array, Bx is an outputvalue of the blue pixel of a respective sub-array and IRx is an outputvalue of the IR pixel of a respective sub-array.

Thus, the values for each color component of each row/column location isdetermined by averaging the respective colored pixels that are adjacentto the particular pixel location or coordinate. For example, the redcolor component for the pixel location r1,c1 is the average of the fourred pixels R1, R2, R3, R4 (at r0,c0; r0,c2; r2,c0 and r2,c2) that areadjacent to the pixel location r1,c1, while the green color componentfor the pixel location r1,c1 is the average of the two green pixels G1,G3 (at r0,c1 and r2,c1) that are adjacent to the pixel location r1,c1,and the blue color component for the pixel location r1,c1 is the averageof the two blue pixels B1, B2 (at r1,c0 and r1,c2) that are adjacent tothe pixel location r1,c1. For the infrared value at the pixel locationr1,c1, the infrared value is determined by the value of the IR1 pixel atthat particular location. The color or IR values for the othercoordinates or pixel locations are determined in a similar manner.

Implementation Methods

It is envisioned that any of the above de-mosaicing techniques may beimplemented in conjunction with any suitable imaging array sensor. Forexample, the above de-mosaicing techniques can be implemented on or inconjunction with an imager chip, which is often called a system-on-chip(SOC). Optionally, such processing techniques may be implemented outsidethe imager chip by means of a FPGA, DSP or microprocessor or the like,while remaining within the spirit and scope of the present invention.

Applications

IHC Reflective Object Identification

In Intelligent Headlamp Control (IHC) applications, spectralinformation, such as R, G, B color information may be used to assistidentifying tail lights, head lamps and traffic signs, such as describedin U.S. Pat. Nos. 5,796,094; 6,097,023; 6,320,176; 6,313,454; 6,559,435;6,831,261; 6,396,397; 5,877,897; 6,498,620; 5,670,935; 6,946,978;7,123,168; 7,004,606; 7,005,974 and/or 5,550,677, and/or PCT ApplicationNo. PCT/US07/75702, filed Aug. 10, 2007 and published Feb. 28, 2008 asInternational Publication No. WO 2008/024639, and/or U.S. patentapplication Ser. No. 11/239,980, filed Sep. 30, 2005, now U.S. Pat. No.7,881,496; and/or Ser. No. 10/534,632, filed May 11, 2005, now U.S. Pat.No. 7,965,336; and/or PCT Application No. PCT/US03/36177, filed Nov. 14,2003 and published Jun. 3, 2004 as International Publication No. WO2004/047421, which are hereby incorporated herein by reference in theirentireties. It is usually challenging to identify and separatereflective objects, like traffic signs in night scene, from tail lights,head lamps and street lights. The techniques that have been often usedare identifying color (traffic signs are often yellow or white) andshapes (traffic signs are often square or rectangle). However, thesetechniques are not always effective and accurate.

The present invention provides a processing technique that uses IRinformation to increase the identification accuracy. In addition tothree visible colors, IR adds one more dimension to the color space. Alltail lights, head lamps and traffic signs have their IR signatures basedon how much they emit or reflect IR light. For instance, a traffic signreflects the light from the head lamps of an approaching vehicle, withthe head lamp illumination carrying a known IR signature. QuantitativeIR signature of objects helps to determine the types of objects, such asfor example, whether an object is a light or a road side traffic sign orother object. The IR signatures include:

-   -   1. Ratios of IR to R signals of object types, IR to G signals of        object types, and IR to B signals of object types.    -   2. An extension to the 2-D CIE color space to form a 3-D space.        The location of an object type (such as a stop sign or the like)        in the 3-D space is a signature that identifies the object type.    -   3. R, G, B, IR 4-D coordinate space. The location of an object        type in the 4-D coordinate space is a signature that identifies        the object type.

In the signatures of the types 2 and 3 above, the vision algorithm cancalibrate and define an area, range or segment of these color spaces toclassify the object types located inside these regions. Thus, the systemmay recognize or identify an object based on whether the signature ofthe detected object falls within a particular signature range that ispreviously categorized or identified as being associated with orindicative of or representative of a particular object or type or classof object.

The system thus may utilize the CIE color space, as is known in thecolor art, such as by utilizing aspects of the systems described in U.S.patent application Ser. No. 11/239,980, filed Sep. 30, 2005, now U.S.Pat. No. 7,881,496, which is hereby incorporated herein by reference inits entirety. The Centre Internationale de l'Eclairage (CIE) is aninternational organization that establishes methods for measuring colorand that recommends standards and procedures for light and lighting,including colorimetry. The color standards for colormetric measurementsare internationally accepted specifications that define color valuesmathematically. CIE defines color as a combination of three axes: x, y,and z. CIE color models are considered device independent because thecolors should not differ, theoretically, from one output device toanother if properly calibrated.

In addition to the spectral domain, one can apply IR processingtechniques to identify an object in the time domain. For example, avehicle can include an IR light source, such as an IR LED or the like,at or near or in the headlamp assembly. The headlamp's IR output can bemodulated with a pulse width modulation (PWM) sequence. Since onlytraffic signs, not the head lamps of other oncoming vehicles or taillights of other leading vehicles reflect the IR light, the traffic signsappear in the image frames that the headlamp IR source is turned on, andis diminished or reduced in the image frames that the headlamp IR sourceis turned off. For the objects seen in the IR image that possibly comefrom oncoming vehicle headlamps, proceeding vehicle tail lights orstreet lights, the captured images of these objects or illuminationsources will not have such an on and off sequence, which is synchronizedwith the IR light source PWM sequence. Such a processing technique canalso help identify and separate the reflective objects in the forwardfield of view of the imager, such as retro-reflective marks/signs thatare placed on the mailbox, front yard, building side or reflectivestripes on trucks. The modulated IR light is not visible or discernibleto humans, so it does not impede drivers of other vehicles or the driverof the subject or host vehicle and does not violate related regulations.The PWM sequence of each vehicle can be encoded or controlled withunique “signature” to avoid the interference from other vehiclesequipped with the same type of PWM IR lights. Optionally, for example,this signature can be one of the following or a combination of thefollowing: (1) unique modulation sequence, (2) unique modulationfrequency.

NIR Night Vision Capability in Forward Viewing Camera

A NIR night vision system of the present invention may include a NIRlight source (such as a LED, laser, or other NIR light sources) thatilluminates the scene in the dark environment, a NIR video camera thatcaptures image/video of the scene, and a display that shows the capturedNIR images to a driver. For example, and with reference to FIG. 8, avision system 110 includes an R-G-B-IR imager or imaging device orcamera 112, a NIR illuminator or illumination source 114 (such as a NIRLED or the like) and a display device 116. The imager may include or maybe associated with a lens or optic 118 for focusing the scene onto animaging plane of the imager. The system 110 includes an R-G-B-IRde-mosaicing processor or algorithm 120 or the like, which provides orgenerates an output, such as for displaying images at the display device116 or for further processing by an associated forward viewing visionsystem 122 (which may be part of or associated with the de-mosaicingprocessor or a separate processor or the like), such as a headlampcontrol system or a lane departure warning system or the like.

The NIR night vision system allows a driver to see a farther distanceahead of the vehicle and to see and discern more objects than he or shecan normally see with typical head lamp illumination. Such a visionsystem thus gives the driver more time to react to different drivingconditions. It is desirable to have a NIR camera mounted inside thevehicle compartment and looking forward. To avoid blocking the viewingangle of driver, it is best to mount the camera behind the rear viewmirror, and preferably with a forward field of view through thewindshield of the vehicle and through an area that is wiped or cleanedby the windshield wipers of the vehicle. The NIR illumination source canbe either mounted inside one or both headlamps or inside the drivercompartment, such as at or in or near or behind the interior rear viewmirror or accessory module or windshield electronics module or the like.

The forward viewing camera thus may be mounted behind the rear viewmirror and inside the vehicle, and the system may provide driverassistance and safety functions, such as intelligent headlamp control,lane departure warning, traffic sign recognition, adaptive cruisecontrol, and/or the like. Optionally, the forward viewing camera mayprovide or be associated with a rain sensing function, or there may beother sensing devices, such as a rain sensor or the like, mounted at ornear or behind the rear view mirror. Adding a NIR camera to an alreadycrowded space behind the rear view mirror can be challenging and costly.It is thus desirable to combine the NIR camera to a visible forwardviewing camera to save space and cost. The R-G-B-IR imager of thepresent invention provides a solution to combine the visible forwardviewing camera and the NIR camera into a single imager and camera body.

Rear Viewing Camera With NIR Night Vision

A typical automotive rear viewing camera includes a color imager and alens with IR cut-off filter. The IR cut-off filter in the lens preventsthe camera from IR contamination and provides good visible color imagesfor the camera in normal lighting conditions. However, in a nightcondition when no or minimum ambient light is available to illuminatethe scene behind the vehicle, the only illumination comes from thevehicle tail lights. The tail lights, which include running lights,brake lights and back up lights, typically produce illumination of lessthan a few Lux. Thus, in a rear viewing camera image with an IR cut-offfilter, the night scene image is mostly noisy, dark and low-contrast.One can thus only view a very short distance behind the vehicle. This ispartially because the IR cut-off filter cuts off a significant amount ofthe existing IR light in the tail light illumination which otherwiseassists the imaging sensitivity.

To provide enhanced safety and enhanced convenience to the drivers, itis desirable to increase the low light sensitivity of the rear viewingcamera. It may be desirable to take off the IR cutoff filter from thelens, so that the scene illuminated by the near IR light component fromthe tail lights, street light, garage light, and/or the like can bebetter imaged by the imager, since incandescent light bulbs, which aretypically used in vehicle tail lights, street lights, garage lights, andthe like, emit a relatively high amount of near IR light. It may also bedesired to include an additional invisible near IR light source (suchas, for example, a near infrared LED or the like) at or near or in thetail light assembly for illuminating the rearward scene of the vehicle.The high sensitivity of silicon imager in the near IR region helpsincrease the image sensitivity of the night scene.

The R-G-B-IR imager described above provides a possibility of using NIRto enhance low light sensitivity of rear viewing cameras. A rear viewingcamera, such as shown in FIG. 9, can provide good color images in normallighting condition and enhanced low light sensitivity in low light ornighttime conditions. In a low light night scene, it is of lessimportance to produce true color images (because human eyes do notdistinguish color as well under low illumination conditions) so ablack-and-white image is sufficient for the night scene display. Ahardware switch can be used to switch the video signal that feeds thedisplay between the normal lighting color image and low light NIR image,such as by utilizing aspects of the imaging systems described in U.S.Pat. No. 6,498,620; and/or U.S. patent application Ser. No. 11/239,980,filed Sep. 30, 2005, now U.S. Pat. No. 7,881,496; and/or Ser. No.10/534,632, filed May 11, 2005, now U.S. Pat. No. 7,965,336; and/or PCTApplication No. PCT/US03/36177, filed Nov. 14, 2003 and published Jun.3, 2004 as International Publication No. WO 2004/047421, which arehereby incorporated herein by reference in their entireties.

For example, and with reference to FIG. 9, a NIR night vision system110′ of the present invention may include an R-G-B-IR imager or imagingdevice or camera 112 and associated lens or optic 118, a NIR illuminatoror illumination source 114 (such as a NIR LED or the like) and a displaydevice 116. The system 110′ includes an R-G-B-IR de-mosaicing processoror algorithm 120′ or the like, which provides or generates an output(such as an RGB color output and an IR output), and the system mayinclude a normal and low light image switch 124 that determines whetheror not to display the captured color images or the captured IR or NIRimages at the display device 116. The switch may adjust the displayedimages in response to the ambient light level at the scene being aboveor below a threshold level.

To further increase the signal strength and sensitivity of the low lightimages, it is desirable to use all the available photons that the imagercan detect in the NIR and visible wavelength ranges. The red and IRpixels are more sensitive to the light sources in low light nightconditions, which includes incandescent bulb tail lights, an added NIRilluminator, garage lamp lights, street lights, and the like, than greenand blue pixels, since most of these lights have stronger red and NIRspectrum range components than green and blue spectrum range components.To take advantage of this feature, a de-mosaicing method of the presentinvention may be implemented to extract the desired image colorcomponents. As can be seen with reference to FIG. 10, the pixel valuesfrom the green and blue pixels are not used for the image processing inthis method. Instead, the combined values coming from the surroundingred and IR pixels are used to represent the pixel values of green andblue pixels. In this method, the imager is treated as a substantiallymonochrome imager. Each pixel produces only one grey scale value.

The switching between normal color image and low light monochrome imagemay be done in an imager de-mosaicing unit. For example, and withreference to FIG. 11, a rear viewing camera system 110″ may include anR-G-B-IR imager or imaging device or camera 112 and associated lens oroptic 118, a NIR illuminator or illumination source 114 (such as a NIRLED or the like) and a display device 116. The system 110″ includes anR-G-B-IR de-mosaicing processor or algorithm 120″, which includes animage mode switch such as described above, and which provides orgenerates an output (such as an RGB color output or color image signaland a monochrome output or monochrome image signal), whereby the displaydevice 116 displays the color image or monochrome image in response tothe de-mosaicing unit.

Optionally, the imaging sensor may include a plurality of sub-arrays,such as a plurality of 2×2 sub-arrays or the like, that may eachcomprise a principally red sensing pixel, a principally blue sensingpixel, a principally green sensing pixel and a principally IR or near IRsensing pixel, as discussed above, or may include other types ofsub-arrays having different arrangements of pixels such as one or moreprincipally IR or NIR sensing pixels and one or more color sensingpixels. For example, an imaging sensor may include a plurality ofsub-arrays where each sub-array may comprise a color sensing pixel (suchas a pixel primarily sensing red light from the red region of thespectrum, blue light from the blue region of the spectrum or green lightfrom the green region of the spectrum, such as may be achieved byplacing a color filter in front of or over the respective pixel) and aclear pixel (that has a clear spectral filter or no spectral filter andis receptive to all or substantially all incoming visible and infraredlight or energy incident thereon) and a principally IR or NIR sensingpixel (such as a principally IR or NIR sensing pixel as describedabove). Such an imaging sensor may function in a similar manner asdescribed above, where the IR value (as sensed by the principally IR orNIR sensing pixel) may be subtracted by the accompanying image processorfrom the color value sensed by the color pixel and the clear valuesensed by the clear pixel to obtain a more accurate or true colorresponse for the color and clear pixels and to limit or avoid infraredcolor wash-out. Clearly, other arrangements of pixels and filters, suchas an imaging sensor having a sub-array having one or more color sensingpixels and/or one or more clear pixels and one or more principally IR orNIR sensing pixels, such as an imaging sensor having three pixelsub-arrays or 2×2 sub-arrays or 3×3 sub-arrays or the like, may beimplemented with one or more IR or NIR sensing pixels while remainingwithin the spirit and scope of the present invention.

A benefit of the present invention is that a principally IR or NIRsensing pixel is in close proximity and optionally contiguous with theassociated cluster of spectrally selective and/or clear pixels (such asred, green and/or blue pixels and/or clear pixels), so that the imagedata associated with a particular object in the field of view beingimaged (such as, for example, a headlamp or taillight in a forward fieldof view or a child or object in a rearward field of view or a vehicle orobject in a forward field of view or a child or seat occupant in aninterior cabin field of view or the like) can be processed to subtractout or account for the IR or NIR content of light or energy in the fieldof view by subtracting out the IR or NIR values (as sensed by theprincipally IR or NIR sensing pixel) from the other pixels of aparticular sub-array of pixels (that images the headlamp or taillight orobject or child or occupant that is in the field of view of the imagingsensor) to enhance the image output and/or to obtain a more accurate ortrue color response for the pixels and to limit or avoid infrared colorwash-out. Thus, the IR or NIR values local to or associated with anillumination source or object being sensed by a respective sub-array ofthe imaging sensor may be subtracted from the color values or clearspectral values sensed by the color and/or clear pixels of thatrespective sub-array (with the color and/or clear pixels of thesub-array being proximate to or associated with the principally IR orNIR sensing pixel or pixels of that sub-array), so that the system mayprovide enhanced detection of and/or identification of the illuminationsources or objects exterior or interior of the vehicle that are beingimaged by the imaging sensor and system.

Optionally, the imaging sensor may be suitable for various forwardimaging systems, such as for headlamp control systems, rain sensorsystems, lane departure warning systems, object detection systems,and/or the like. For example, the imaging sensor of the presentinvention may be associated with one or more systems of the typesdescribed in U.S. Pat. Nos. 5,796,094; 6,396,397; 6,806,452; 6,690,268;7,005,974; 7,123,168; 7,004,606; 6,946,978; 7,038,577; 6,353,392;6,320,176; 6,313,454 and 6,824,281, which are all hereby incorporatedherein by reference in their entireties.

Optionally, the imaging sensor may be suitable for various in-cabinetimaging systems, such as driver drowsiness monitoring, passengeridentification for intelligent airbag deployment, back seat passengermonitoring, and/or the like.

Optionally, the imaging sensor may be suitable for use in connectionwith other vehicle imaging systems, such as, for example, a blind spotdetection system, where a blind spot indicator may be operable toprovide an indication to the driver of the host vehicle that an objector other vehicle has been detected in the lane or area adjacent to theside of the host vehicle. In such a blind spot detector/indicatorsystem, the blind spot detection system may include an imaging sensor orsensors, or ultrasonic sensor or sensors, or sonar sensor or sensors orthe like. For example, the blind spot detection system may utilizeaspects of the blind spot detection and/or imaging and/or indicatingsystems described in U.S. Pat. Nos. 7,038,577; 6,882,287; 6,198,409;5,929,786 and/or 5,786,772, and/or U.S. pat. application Ser. No.11/315,675, filed Dec. 22, 2005, now U.S. Pat. No. 7,720,580; Ser. No.11/239,980, filed Sep. 30, 2005, now U.S. Pat. No. 7,881,496; and/orSer. No. 11/933,697, filed Nov. 1, 2007, now U.S. Pat. No. 7,777,611,and/or PCT Application No. PCT/US06/26148, filed Jul. 5, 2006 andpublished Jan. 11, 2007 as International Publication No. WO 2007/005942,and/or PCT Application No. PCT/US07/082099, filed Oct. 22, 2007 andpublished May 2, 2008 as International Publication No. WO 2008/051910,and/or U.S. provisional applications, Ser. No. 60/618,686, filed Oct.14, 2004; Ser. No. 60/853,850, filed Oct. 24, 2006; Ser. No. 60/918,089,filed Mar. 15, 2007; Ser. No. 60/970,687, filed Sep. 7, 2007; and/orSer. No. 60/857,025, filed Nov. 6, 2006, and/or of the reverse or backupaid systems, such as the rearwardly directed vehicle vision systemsdescribed in U.S. Pat. Nos. 5,550,677; 5,760,962; 5,670,935; 6,201,642;6,396,397; 6,498,620; 6,717,610; 6,757,109 and/or 7,005,974, and/or ofthe rain sensors described in U.S. Pat. Nos. 6,250,148 and 6,341,523,and/or of other imaging systems, such as the types described in U.S.Pat. Nos. 7,123,168; 6,353,392 and 6,313,454, with all of the abovereferenced U.S. patents, patent applications and provisionalapplications and PCT applications being commonly assigned and beinghereby incorporated herein by reference in their entireties.

Optionally, the mirror assembly and/or accessory module or windshieldelectronics module may include one or more displays, such as fordisplaying the captured images or video images captured by the imagingsensor or sensors of the vehicle, such as the displays of the typesdisclosed in U.S. Pat. Nos. 7,004,593; 5,530,240 and/or 6,329,925, whichare hereby incorporated herein by reference, and/or display-on-demand ortransflective type displays, such as the types disclosed in U.S. Pat.Nos. 7,274,501; 7,255,451; 7,195,381; 7,184,190; 6,690,268; 5,668,663and/or 5,724,187, and/or in U.S. patent application Ser. No. 10/538,724,filed Jun. 13, 2005 and published Mar. 9, 2006 as U.S. Publication No.US 2006-0050018; Ser. No. 11/226,628, filed Sep. 14, 2005 and publishedMar. 23, 2006 as U.S. Publication No. US 2006-0061008; Ser. No.10/993,302, filed Nov. 19, 2004, now U.S. Pat. No. 7,338,177; and/orSer. No. 11/284,543, filed Nov. 22, 2005, now U.S. Pat. No. 7,370,983,and/or PCT Application No. PCT/US03/29776, filed Sep. 9, 2003 andpublished Apr. 1, 2004 as International Publication No. WO 2004/026633;and/or PCT Application No. PCT/US03/35381, filed Nov. 5, 2003 andpublished May 21, 2004 as International Publication No. WO 2004/042457,and/or PCT Application No. PCT/US03/40611, filed Dec. 19, 2003 andpublished Jul. 15, 2004 as International Publication No. WO 2004/058540,and/or PCT Application No. PCT/US06/18567, filed May 15, 2006 andpublished Nov. 23, 2006 as International Publication No. WO 2006/124682,which are all hereby incorporated herein by reference in theirentireties, or may include or incorporate video displays or the like,such as the types described in PCT Application No. PCT/US03/40611, filedDec. 19, 2003 and published Jul. 15, 2004 as International PublicationNo. WO 2004/058540, and/or PCT Application No. PCT/US07/042718, filedOct. 31, 2006 and published May 10, 2007 as International PublicationNo. WO 07/053710, and/or U.S. Pat. application Ser. No. 10/538,724,filed Jun. 13, 2005 and published Mar. 9, 2006 as U.S. Publication No.US 2006-0050018; and/or Ser. No. 11/284,543, filed Nov. 22, 2005, nowU.S. Pat. No. 7,370,983, and/or U.S. provisional applications, Ser. No.60/732,245, filed Nov. 1, 2005; Ser. No. 60/759,992, filed Jan. 18,2006; and/or Ser. No. 60/836,219, filed Aug. 8, 2006, which are herebyincorporated herein by reference.

The imaging sensor may be incorporated at or in an accessory module orwindshield electronics module (such as described above), or may beincorporated at or in an interior rearview mirror assembly of thevehicle, while remaining within the spirit and scope of the presentinvention. Optionally, the accessory module may utilize aspects of otheraccessory modules or windshield electronics modules or the like, such asthe types described in U.S. patent application Ser. No. 11/201,661,filed Aug. 11, 2005, and/or U.S. Pat. Nos. 7,188,963; 7,004,593;6,824,281; 6,690,268; 6,250,148; 6,341,523; 6,593,565; 6,428,172;6,501,387; 6,329,925 and 6,326,613, and/or in PCT Application No.PCT/US03/40611, filed Dec. 19, 2003 and published Jul. 15, 2004 asInternational Publication No. WO 2004/058540, and/or Ireland pat.applications, Ser. No. S2004/0614, filed Sep. 15, 2004; Ser. No.S2004/0838, filed Dec. 14, 2004; and Ser. No. S2004/0840, filed Dec. 15,2004, which are all hereby incorporated herein by reference.

Optionally, the mirror assembly and/or module may support one or moreother accessories or features, such as one or more electrical orelectronic devices or accessories. For example, illumination sources orlights, such as map reading lights or one or more other lights orillumination sources, such as illumination sources of the typesdisclosed in U.S. Pat. Nos. 6,690,268; 5,938,321; 5,813,745; 5,820,245;5,673,994; 5,649,756; 5,178,448; 5,671,996; 4,646,210; 4,733,336;4,807,096; 6,042,253; 6,971,775; 7,249,860; 7,195,381 and/or 5,669,698,which are hereby incorporated herein by reference in their entireties,may be included in the mirror assembly or module. The illuminationsources and/or the circuit board may be connected to one or more buttonsor inputs for activating and deactivating the illumination sources.Optionally, the mirror assembly or module may also or otherwise includeother accessories, such as microphones, such as analog microphones ordigital microphones or the like, such as microphones of the typesdisclosed in U.S. Pat. Nos. 6,243,003; 6,278,377 and/or 6,420,975,and/or in U.S. patent application Ser. No. 10/529,715, filed Mar. 30,2005. Optionally, the mirror assembly may also or otherwise includeother accessories, such as a telematics system, speakers, antennas,including global positioning system (GPS) or cellular phone antennas,such as disclosed in U.S. Pat. No. 5,971,552, a communication module,such as disclosed in U.S. Pat. No. 5,798,688, a voice recorder,transmitters and/or receivers, such as for a garage door opener or avehicle door unlocking system or the like (such as a remote keylessentry system), a digital network, such as described in U.S. Pat. No.5,798,575, a memory mirror system, such as disclosed in U.S. Pat. No.5,796,176, a hands-free phone attachment, a video device for internalcabin surveillance (such as for sleep detection or driver drowsinessdetection or the like) and/or video telephone function, such asdisclosed in U.S. Pat. Nos. 5,760,962 and/or 5,877,897, a remote keylessentry receiver, a seat occupancy detector, a remote starter control, ayaw sensor, a clock, a carbon monoxide detector, status displays, suchas displays that display a status of a door of the vehicle, atransmission selection (4wd/2wd or traction control (TCS) or the like),an antilock braking system, a road condition (that may warn the driverof icy road conditions) and/or the like, a trip computer, a tirepressure monitoring system (TPMS) receiver (such as described in U.S.Pat. Nos. 6,124,647; 6,294,989; 6,445,287; 6,472,979 and/or 6,731,205;and/or U.S. patent application Ser. No. 11/232,324, filed Sep. 21, 2005,now U.S. Pat. No. 7,423,522, and/or an ONSTAR® system and/or any otheraccessory or circuitry or the like (with all of the above-referencedpatents and PCT and U.S. patent applications being commonly assigned toDonnelly Corporation, and with the disclosures of the referenced patentsand patent applications being hereby incorporated herein by reference intheir entireties).

Changes and modifications to the specifically described embodiments maybe carried out without departing from the principles of the presentinvention, which is intended to be limited by the scope of the appendedclaims, as interpreted in accordance with the principles of patent law.

The invention claimed is:
 1. A vehicular imaging system, said vehicularimaging system comprising: a camera comprising an imaging sensor;wherein said camera is configured for attachment at an in-cabin side ofa windshield of a vehicle equipped with said vehicular imaging system,and wherein, with said camera attached at the in-cabin side of thewindshield of the equipped vehicle, said camera has a field of viewforward of the equipped vehicle through the windshield of the equippedvehicle; wherein said imaging sensor of said camera comprises atwo-dimensional array of photosensing pixels; wherein a first spectralfilter is disposed at each photosensing pixel of a first set ofphotosensing pixels of said two-dimensional array, whereby said firstset of photosensing pixels primarily senses a first spectral band ofvisible light; wherein a second spectral filter is disposed at eachphotosensing pixel of a second set of photosensing pixels of saidtwo-dimensional array, whereby said second set of photosensing pixelsprimarily senses a second spectral band of visible light; wherein eachphotosensing pixel of a third set of photosensing pixels of saidtwo-dimensional array senses near infrared radiation; wherein saidimaging sensor of said camera captures image data, and wherein imagedata captured by said imaging sensor is provided to an image processor;wherein the image data provided to said image processor comprises outputof said first set of photosensing pixels of said two-dimensional array,output of said second set of photosensing pixels of said two-dimensionalarray and output of said third set of photosensing pixels of saidtwo-dimensional array; wherein said image processor utilizesde-mosaicing when processing outputs of photosensing pixels ofindividual sub-arrays of said two-dimensional array of photosensingpixels, each sub-array of the individual sub-arrays includingphotosensing pixels of said first, second and third sets of photosensingpixels; wherein the de-mosaicing comprises reducing the outputs of thephotosensing pixels of said first and second sets of photosensing pixelsbased on the outputs of the photosensing pixels of said third set ofphotosensing pixels; wherein the outputs of the photosensing pixels ofsaid first and second sets of photosensing pixels of saidtwo-dimensional array of photosensing pixels are processed by said imageprocessor for a color camera function of the equipped vehicle; andwherein the output of the photosensing pixels of said third set ofphotosensing pixels of said two-dimensional array of photosensing pixelsis processed by said image processor for a night vision function of theequipped vehicle.
 2. The vehicular imaging system of claim 1, whereinoutputs of the photosensing pixels of said first and second sets ofphotosensing pixels of said two-dimensional array of photosensing pixelsare processed for at least one selected from the group consisting of (i)a lane departure warning system of the equipped vehicle and (ii) anobject detection system of the equipped vehicle.
 3. The vehicularimaging system of claim 1, wherein said first set of photosensing pixelsprimarily senses red visible light.
 4. The vehicular imaging system ofclaim 3, wherein the second spectral filter comprises a clear filter. 5.The vehicular imaging system of claim 1, wherein a fourth spectralfilter is disposed at each photosensing pixel of a fourth set ofphotosensing pixels of said two-dimensional array, whereby said fourthset of photosensing pixels primarily senses a fourth spectral band ofvisible light.
 6. The vehicular imaging system of claim 5, wherein thesecond spectral filter and the fourth spectral filter comprise clearfilters.
 7. The vehicular imaging system of claim 6, wherein said firstset of photosensing pixels primarily senses red visible light.
 8. Thevehicular imaging system of claim 5, wherein the first spectral band ofvisible light is different than the second spectral band of visiblelight and the fourth spectral band of visible light, and wherein thesecond spectral band of visible light is different than the fourthspectral band of visible light.
 9. The vehicular imaging system of claim1, comprising a near infrared illumination source disposed at a frontportion of the equipped vehicle, and wherein said near infraredillumination source, when operated, illuminates at least a portion ofthe field of view of said camera with non-visible near infraredillumination.
 10. The vehicular imaging system of claim 9, wherein saidnear infrared illumination source is disposed in a headlamp assembly ofthe equipped vehicle.
 11. The vehicular imaging system of claim 10,wherein said near infrared illumination source, when operated, emitsnon-visible near infrared illumination that is modulated by pulse widthmodulation (PWM).
 12. The vehicular imaging system of claim 11, whereinimage data capture by said imaging sensor is synchronized withmodulation of the non-visible near infrared illumination emitted by saidnear infrared illumination source of the headlamp assembly.
 13. Thevehicular imaging system of claim 11, wherein the non-visible nearinfrared illumination emitted by said near infrared illumination sourceof the headlamp assembly is encoded with a modulation sequence unique tothe equipped vehicle to avoid other vehicles traveling ahead of theequipped vehicle interfering with said vehicular imaging system.
 14. Thevehicular imaging system of claim 9, wherein said near infraredillumination source comprises a near infrared light emitting diode. 15.The vehicular imaging system of claim 9, wherein said near infraredillumination source comprises a near infrared emitting laser.
 16. Thevehicular imaging system of claim 1, wherein said vehicular imagingsystem is operable to generate color video images based on capturedimage data.
 17. The vehicular imaging system of claim 1, wherein saidvehicular imaging system is associated with an intelligent headlampcontrol system of the equipped vehicle.
 18. The vehicular imaging systemof claim 1, wherein, responsive at least in part to processing by saidimage processor of output of said first set of photosensing pixels andwith said camera attached at the in-cabin side of the windshield of theequipped vehicle and viewing forward through the windshield of theequipped vehicle, said vehicular imaging system is operable to detecttraffic signs.
 19. The vehicular imaging system of claim 1, wherein,responsive at least in part to processing by said image processor ofimage data captured by said imaging sensor and with said camera attachedat the in-cabin side of the windshield of the equipped vehicle andviewing forward of the equipped vehicle, said vehicular imaging systemis operable to detect objects located external of the equipped vehicle.20. A vehicular imaging system, said vehicular imaging systemcomprising: a camera comprising an imaging sensor; wherein said camerais configured for attachment at an in-cabin side of a windshield of avehicle equipped with said vehicular imaging system, and wherein, withsaid camera attached at the in-cabin side of the windshield of theequipped vehicle, said camera has a field of view forward of theequipped vehicle through the windshield of the equipped vehicle; whereinsaid imaging sensor of said camera comprises a two-dimensional array ofphotosensing pixels; wherein a first spectral filter is disposed at eachphotosensing pixel of a first set of photosensing pixels of saidtwo-dimensional array, whereby said first set of photosensing pixelsprimarily senses red visible light; wherein a second spectral filter isdisposed at each photosensing pixel of a second set of photosensingpixels of said two-dimensional array, whereby said second set ofphotosensing pixels primarily senses visible light; wherein eachphotosensing pixel of a third set of photosensing pixels of saidtwo-dimensional array senses near infrared radiation; wherein saidimaging sensor of said camera captures image data, and wherein imagedata captured by said imaging sensor is provided to an image processor;wherein the image data provided to said image processor comprises outputof said first set of photosensing pixels of said two-dimensional array,output of said second set of photosensing pixels of said two-dimensionalarray and output of said third set of photosensing pixels of saidtwo-dimensional array; wherein said image processor utilizesde-mosaicing when processing outputs of photosensing pixels ofindividual sub-arrays of said two-dimensional array of photosensingpixels, each sub-array of the individual sub-arrays includingphotosensing pixels of said first, second and third sets of photosensingpixels; wherein the de-mosaicing comprises reducing the outputs of thephotosensing pixels of said first and second sets of photosensing pixelsbased on the outputs of the photosensing pixels of said third set ofphotosensing pixels; wherein the outputs of the photosensing pixels ofsaid first and second sets of photosensing pixels of saidtwo-dimensional array of photosensing pixels are processed by said imageprocessor for at least one selected from the group consisting of (i) alane departure warning system of the equipped vehicle and (ii) an objectdetection system of the equipped vehicle; and wherein the output of thephotosensing pixels of said third set of photosensing pixels of saidtwo-dimensional array of photosensing pixels is processed by said imageprocessor for a night vision function of the equipped vehicle.
 21. Thevehicular imaging system of claim 20, comprising a near infraredillumination source disposed at a front portion of the equipped vehicle,and wherein said near infrared illumination source, when operated,illuminates at least a portion of the field of view of said camera withnon-visible near infrared illumination.
 22. A vehicular imaging system,said vehicular imaging system comprising: a camera comprising an imagingsensor; wherein said camera is configured for attachment at an in-cabinside of a windshield of a vehicle equipped with said vehicular imagingsystem, and wherein, with said camera attached at the in-cabin side ofthe windshield of the equipped vehicle, said camera has a field of viewforward of the equipped vehicle through the windshield of the equippedvehicle; wherein said imaging sensor of said camera comprises atwo-dimensional array of photosensing pixels; wherein a first spectralfilter is disposed at each photosensing pixel of a first set ofphotosensing pixels of said two-dimensional array, whereby said firstset of photosensing pixels primarily senses a first spectral band ofvisible light; wherein a second spectral filter is disposed at eachphotosensing pixel of a second set of photosensing pixels of saidtwo-dimensional array, whereby said second set of photosensing pixelsprimarily senses a second spectral band of visible light; wherein eachphotosensing pixel of a third set of photosensing pixels of saidtwo-dimensional array senses near infrared radiation; wherein a fourthspectral filter is disposed at each photosensing pixel of a fourth setof photosensing pixels of said two-dimensional array, whereby saidfourth set of photosensing pixels primarily senses a fourth spectralband of visible light; wherein said imaging sensor of said cameracaptures image data, and wherein image data captured by said imagingsensor is provided to an image processor; wherein the image dataprovided to said image processor comprises output of said first set ofphotosensing pixels of said two-dimensional array, output of said secondset of photosensing pixels of said two-dimensional array and output ofsaid third set of photosensing pixels of said two-dimensional array;wherein said image processor utilizes de-mosaicing when processingoutputs of photosensing pixels of individual sub-arrays of saidtwo-dimensional array of photosensing pixels, each sub-array of theindividual sub-arrays including photosensing pixels of said first,second and third sets of photosensing pixels; wherein the de-mosaicingcomprises reducing the outputs of the photosensing pixels of said firstand second sets of photosensing pixels based on the outputs of thephotosensing pixels of said third set of photosensing pixels; whereinthe outputs of the photosensing pixels of said first and second sets ofphotosensing pixels of said two-dimensional array of photosensing pixelsare processed by said image processor for at least one selected from thegroup consisting of (i) a lane departure warning system of the equippedvehicle and (ii) an object detection system of the equipped vehicle; andwherein the output of the photosensing pixels of said third set ofphotosensing pixels of said two-dimensional array of photosensing pixelsis processed by said image processor for a night vision function of theequipped vehicle.
 23. The vehicular imaging system of claim 22, whereinsaid first set of photosensing pixels primarily senses red visiblelight.
 24. The vehicular imaging system of claim 23, wherein the secondspectral filter and the fourth spectral filter comprise clear filters.25. The vehicular imaging system of claim 23, wherein said second set ofphotosensing pixels primarily senses blue visible light and said fourthset of photosensing pixels primarily senses green visible light.
 26. Thevehicular imaging system of claim 22, comprising a near infraredillumination source disposed at a front portion of the equipped vehicle,and wherein said near infrared illumination source, when operated,illuminates at least a portion of the field of view of said camera withnon-visible near infrared illumination.